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Abstract 

We construct a probabilistic coherence measure 
for information sets which determines a partial 
coherence ordering. This measure is applied in 
constructing a criterion for expanding our beliefs 
in the face of new information. A number of ide- 
alizations are being made which can be relaxed 
by an appeal to Bayesian Networks. 



Introduction 

Suppose that one receives information {Ri,...,R n } 
from n independent but less than fully reliable sources. 
Is it rational to believe this information? Following a 
tradition in epistemology that goes back to John Locke, 
we let belief correspond to a sufficiently high degree of 
confidence (Foley 1992, Hawthorne and Bovens 1999). 
There are three factors that determine this degree of 
confidence: (i) How surprising is the information? (ii) 
How reliable are the sources? (iii) How coherent is 
the information? First, suppose that the sources are 
halfway reliable and the information is halfway coher- 
ent. Then certainly the degree of confidence will be 
greater when the reported information is less rather 
than more surprising. Second, suppose that the in- 
formation is halfway surprising and is halfway coher- 
ent. Let truth-tellers provide fully reliable information 
and let randomizers flip a coin for each proposition to 
determine whether they will affirm or deny it. Then 
certainly the degree of confidence will be greater when 
the sources are more like truth-tellers than when they 
are more like randomizers. Third, consider the follow- 
ing story: a scientist runs two independent tests to 
determine the locus of a genetic disease on the human 
genome. In the first case, the tests respectively point to 
two fairly narrow regions that just about overlap in a 
particular region. In the second case, the tests respec- 
tively point to fairly broad regions that have minimal 
overlap in the very same region. Suppose that the tests 
are halfway reliable and that this region is a somewhat 
surprising locus for the disease. Then certainly the de- 
gree of confidence that the locus of the disease is in 
this region is greater in the former case, in which the 



information is more coherent, than in the latter case, 
in which the information is less coherent. 

We define measures for each of these determinants of 
the degree of confidence in a probabilistic framework. 
The real challenge lies in developing a measure of co- 
herence (cf. Lewis 1946, Bonjour 1985). This measure 
defines a partial ordering over information sets. Sub- 
sequently, we argue that belief expansion is a function 
of the reliability of the sources and the coherence of 
the new information with the information that we al- 
ready believe. We construct an acceptance measure 
which determines whether newly acquired information 
can be added to our beliefs under alternative supposi- 
tions about the reliability of the sources. Our calcu- 
lations rest on some results in the theory of Bayesian 
Networks. Throughout we have made some strong ide- 
alizations. We show how these idealizations can be 
relaxed by directly invoking Bayesian Networks. 

The Model 

For each proposition R; (in roman script) in the in- 
formation set, let us define a propositional variable 
Ri (in italic script) which can take on two values, 
viz. R; and Ri (i.e. not-Ri) for i = l,...,n. Let 
REPRi be a propositional variable which can take on 
two values, viz. REPR;, i.e. there is a report from 
the proper source to the effect that Ri is true, and 
REPRi, i.e. there is a report to the effect that Ri 
is false. We construct a joint probability distribution 
P over Rx,...,Rn, REPRi, REPR n satisfying the 
constraint that the sources are independent and less 
than fully reliable. 

We model independence by stipulating that P re- 
spects the following conditional independences: 

REPR, _L Rj , REPR^R, iori^j; i, j = 1, 2, . . . , n 

or, in words, REPRi is probabilistically independent 
of Rj, REPRj, given Ri, for i ^ j and i, j = 1, 2, . . . , n. 
What this means is that the probability that I will 
receive a report that Ri given that Ri is the case or 
given that R; is not the case, is not affected by any 
additional information about whether Rj is the case or 
whether there is a report to the effect that Rj is the 



case. Each source tunes in on the item of information 
that it is meant to report on: it may not always provide 
an accurate report, but its report is not affected by 
what other sources have to report or by other items 
of information than the one it reports on (Lewis 1946, 
B ovens and Olsson 1999). 

We define a less-than- fully-reliable source as a source 
that is better than a randomizer, but short of being a 
truth-teller and make the simplifying idealization that 
the information sources are equally reliable. We spec- 
ify the following two parameters: P(REPRj|Ri) = p 
and P(REPRi|Ri) = q for i = 1, . . . , n. If the informa- 
tion sources are truth-tellers, then p = 1 and q = 0, 
while if they are randomizers, then p = q > 0. We 
model less-than-full-reliability by imposing the follow- 
ing constraint on P: 

P(REPRi|Ri) = p > q = P(REPRi|Ri) > (2) 

The degree of confidence in the content of the infor- 
mation set is the posterior joint probability after all 
the reports have come in: 

P*(Ri, . . . ,R n ) = P(Ri, . . . ,R„|REPRi, . . . ,REPR n ) 

The motivation for the definition of less-than-full re- 
liability is that we are interested in cases in which in- 
coming information raises our confidence in the con- 
tent of the information set to different levels. When 
the sources are randomizers, our confidence will be 
unaffected (Huemer 1997, Bovens and Olsson 1999), 
i.e. P*(Ri,...,R n ) = P(Ri,...,R n ); when they 
are truth-tellers, our confidence will be raised to cer- 
tainty, i.e. P*(Ri, . . . , R n ) = 1; and when they are 
worse than randomizers, our confidence will drop, i.e. 
P*(Ri,...,R n ) < P(Ri,...,R n ). 

Expectation, Reliability and Coherence 

It can be shown by the probability calculus, that, given 
the constraints on P in ([[]) and (g), 
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in which the likelihood ratio x — q/p (note that 
< x < 1 for p > q > 0) and a* is the sum 
of the joint probabilities of all combinations of val- 
ues of the variables Pi , . . . , R n that have i negative 
values and n — i positive values: e. g. for n = 3, 
a 2 = P(R 1 ,R 2) R 3 ) + P(Ri,R 2 ,R 3 ) + P(Ri,R 2 ,R 3 ). 
Note that Ya=o °» = !• 

We can directly identify the first determinant of the 
degree of confidence in the information set. Note that 
ciq = P(Ri , . . . , R n ) is the prior joint probability of the 
propositions in the information set, i.e. the probabil- 
ity before any information was received. This prior 
probability is lower for more surprising information 
and higher for less surprising information. Since more 
surprising information is tantamount to less expected 
information, let us call this prior probability the expec- 
tation measure. It is easy to see that P*(Ri, . . . , R n ) is 



a monotonically increasing function of oq. We can also 
directly identify the second determinant, i.e. the reli- 
ability of the sources. Note that P*(Ri, . . . ,R n ) is a 
monotonically decreasing function of x = q/p. Hence, 
let us call r :— 1 — x the reliability measure, since 
P*(Ri, . . . ,R n ) is a monotonically increasing function 
of r and this measure ranges from for sources that 
are randomizers to 1 for sources that are truth-tellers. 

It is more difficult to construct a coherence measure. 
Consider the following analogy: to assess the impact 
of a training program, we consider the rate of the stu- 
dent's actual performance level over the performance 
level that he would have reached in an ideal training 
program, all other things equal. Similarly, to assess the 
impact of coherence, we consider the rate of the present 
degree of confidence over the degree of confidence that 
would have been obtained had the information set been 
maximally coherent, all other things equal. The infor- 
mation set would have been maximally coherent if and 
only if Ri, . . . , R n had all been coextensive. Let P be 
the actual joint probability distribution. Construct a 
joint probability distribution p max with the same ex- 
pectation measure and the same reliability measure as 
P, but Ri,...,R n are all coextensive, i.e., on p max i 
oq is the same as on P, but a n — 1 — ag =: So, so that 
ai = 0, for all i ^ 0,n. It follows from (0) that, 
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is a measure of the impact of the coherence of the infor- 
mation set on the degree of confidence in the content 
of the information set. But note that this measure is 
contingent on the value of the reliability measure: (^|) 
only provides us with a reliability-relative coherence 
measure. This is unwelcome: there is a pretheoretical 
notion of the coherence of an information set which has 
nothing to do with the reliability of the sources that 
provides us with their content. On the other hand, 
this pretheoretical notion seems to be an ordinal rather 
than a cardinal notion. And furthermore, it seems to 
require a partial rather than a complete ordering over 
information sets: for certain, though not for all pairs of 
information sets, we are prepared to pass a judgment 
that one set in the pair is more or less coherent than 
the other. 

It turns out that the reliability-relative coherence 
measure indeed induces a partial ordering over infor- 
mations sets which is not contingent on the reliabil- 
ity of the sources. Consider two information sets of 
size n. These sets can be represented by the marginal 
probability distributions P and P' over Pi, ... , R n . It 
can be shown that for some P with (oq, . . . , a„) and 



P' with (a' , . . . , a' n ), the difference c x (R\, . . . , R n ) — 
c' x (R\, . . . ,R n ) has the same sign for any value of x 
ranging from to 1. Hence, the reliability-relative co- 
herence measure c x (R\, . . . ,R n ) induces a partial co- 
herence ordering over information sets that is not con- 
tingent on the reliability of the sources. For infor- 
mation pairs, i.e. for information sets containing ex- 
actly two propositions, it can be shown that the fol- 
lowing is a necessary and sufficient condition for in- 
clusion in the partial coherence ordering: P and P' 
are such that (i) ao/a' < a\ja\ and a\ > a[, or, (ii) 
ao/flo — a i/ a i an d ai < a'i- For information sets in 
general, it can be shown that the following is a suf- 
ficient condition for inclusion in the partial coherence 
ordering: P and P' are such that (i) ai/a^ < ao/a' < 1, 
or, (ii) di/di > a /a' > 1, for i = 1, . . . , n — 1. 

We provide an example of this condition for infor- 
mation pairs. Suppose that we are trying to locate a 
corpse of a murder somewhere in Tokyo. We draw a 
grid of 100 squares over the map of the city so that it 
is equally probable that the murder occurred in each 
grid. We interview two independent less-than-fully- 
reliable sources. Source 1 reports that the corpse is 
somewhere in squares 41 to 60 and source 2 reports 
that the corpse is somewhere in squares 51 to 70. In 
this case, a = .10 and a\ = .20. This our base case. 
Now consider alternate case A in which source 1 re- 
ports squares 50 to 60 and source 2 reports squares 51 
to 61. In this case, a' = .10 and — .02. The infor- 
mation set in alternate case A is clearly more coherent 
than in the base case. Notice that the condition for a 
partial ordering is indeed satisfied. But now consider 
alternate case B: source 1 reports squares 26 to 60 and 
source 2 reports squares 41 to 75. In this case cl'q = .20 
and a" — .30. Is the information set in alternate case B 
more coherent than in the base case? The proportion 
of the reported squares that overlap in each report is 
greater in the alternate case, which suggests that there 
is more coherence. But on the other hand, the price 
of getting more proportional overlap is that the over- 
lapping area is less precise and that both sources make 
a much broader sweep over the map, suggesting less 
coherence. Indeed, in this case, we cannot pass judg- 
ment whether the information set in alternate case B 
is more coherent than in the base case. Notice that the 
condition for a partial ordering is indeed not justified. 

Belief Expansion 

Suppose that we acquire various items of background 
information from various sources and that our degree 
of confidence in the content of the information set is 
sufficiently high to believe the information. Now a new 
item of information is being presented. Are we justified 
to add this new item of information to what we already 
believe? The answer to this question has something to 
do (i) with the reliability of the information source as 
well as (ii) with the plausibility of the new informa- 
tion, given what we already believe, or in other words, 



with how well the new information coheres with the 
background information. The more reliable the source 
is, the less plausible the new information needs to be, 
given what we already believe, to be justified to add 
the new information. The more plausible the new in- 
formation is, given what we already believe, the less 
reliable the source needs to be, to be justified to add 
the new information. The challenge is: can a precise 
account of this relationship be provided? 

Our approach is markedly different from AGM be- 
lief revision. In the AGM approach, the question is not 
whether to accept new information or not, but rather, 
once we have made the decision to accept the new in- 
formation, how we should revise our beliefs in the face 
of inconsistency (Makinson 1997, Olsson 1997). Our 
approach shares a common motivation with the pro- 
gram of non-prioritized belief revision. According to 
Hansson (1997), we may not be willing to accept the 
new information because "it may be less reliable (...) 
than conflicting old information." Makinson (1997) 
writes that "we may not want to give top priority to 
new information (...) we may wish to weigh it against 
old material, and if it is really just too far-fetched or 
incredible, we may not wish to accept it." However, 
whereas the program of non-prioritized belief revision 
operates within a logicist framework, we construct a 
probabilistic model. The cost of this approach is that 
it is informationally more demanding. The benefit is 
that it is empirically more adequate, because it is sensi- 
tive to degrees of reliability and coherence and to their 
interplay in belief acceptance. In non-prioritized belief 
revision, the reliability of the sources does not enter 
into the model itself and the lack of coherence of an 
information set is understood in terms of logical in- 
consistency, which is only a limiting case in our model. 
To introduce the approach, we address the question of 
belief expansion. We believe that our model also car- 
ries a promise to handle belief revision in general, but 
this project is beyond the scope of this paper. 

We need to make some simplifying assumptions 
about the origin of the background information and 
the new information: (a) the propositions in the 
background information arc provided by independent 
sources, which are (b) less than fully reliable, (c) 
equally reliable as the new source, and (d) indepen- 
dent of the new source. 

Our background information is contained in 
{Ri,...,R n }. Now suppose that we have a certain 
threshold level for belief and that the degree of con- 
fidence for the background information after having 
received a report to this effect from independent less 
than fully reliable sources is right at this level. (This 
stipulation is not required if we model actual cases by 
means of Bayesian Networks.) Now we are handed a 
new item of information R n+ i by an independent less 
than fully reliable source. Then we will expand our 
belief set from {Ri, . . . , R n } to {Ri, . . . , R n +i} if and 



only if 

P(Ri 



. . . , R-n+i REPRi, . . . , REPR n -|-i) > 
P(Ri, . . . ,R„|REPRi, . . . , REPR n ). 



(7) 



Our sources are independent: 



REPRi 1 Rj , REPR 3 \ Ri for i ^ j; i,j = l,...,n+l 

^ . . (8) 

(H) defines an acceptance measure for an information 

set: 



e^Ri, . . . , R m ) — P*(Ri, . . . , R m ) 
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Considering (g) and (Q), we can define this accep- 
tance measure in terms of the reliability-relative co- 
herence measure c x , provided that ap ^ 0: 



e x (Ri, . . . , R m ) — 
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-c*(Ri,...,R m ) (10) 



; From (0) and (||), it follows that we can expand our 
belief set with a new item of information if and only if 

e x (Ri,...,R n+ i) > e x (Ri, . . . , R n ). (11) 

We can make the following two observations: 

(i) From (|9|) and (p]), it is clear that whether we can 
expand our beliefs or not, is a complex function of 
the reliability of the sources and the dependence of 
new on earlier information as expressed in the prob- 
ability distribution over the variables R%, . . . , P ra +i- 
The reliability of the sources is reflected in the like- 
lihood ratio x and the dependence of new on earlier 
information is reflected in the series (ao, . . . , a n ) for 
e x (Ri, . . . , R n ) and in the series (a' , . . . , a' n+1 ) for 
e a (Ri, . . . , R n +i). 

(ii) From (|l^), it is clear that the acceptance mea- 
sure is a weighted reliability-relative coherence mea- 
sure. The weight tends to 1 for smaller values of x, 
i.e. for more reliable sources, and for greater val- 
ues of n, i.e. for larger information sets, so that 
the acceptance measure will coincide with c x . We 
have shown that this measure lets us construct a 
coherence ordering over a pair of information n- 
tuples, if certain conditions are met. We conjec- 
ture that such an ordering can also be constructed 
over pairs containing an information n-tuple and 
an expansion of this n-tuple, i.e. over pairs of the 
form {{Ri, . . . , R n }, {Ri, ■ ■ ■ j Rn+i}}, if certain con- 
ditions are met. Contingent on this conjecture, we 
can make a substantial point: if there exists a deter- 
minate answer to the relative coherence of the old 
and the new information sets, then the more reliable 
the sources are and the larger the information set is, 
the more the question of belief expansion is deter- 
mined by whether the new information set is or is 
not more coherent than the old information set, and 
not by the reliability of the sources. 



The acceptance measure depends, at least to some 
extent, on the value of the likelihood ratio x. But 
what, one might ask, should we do when we have no 
clue whatsoever about the reliability of the sources, 
except that they are better than mere randomizers and 
yet less than fully reliable? Let us model our limited 
knowledge as a uniform distribution over the values p 
and q under the constraint that p > q. Then we can 
construct the following averaged acceptance measure: 



dqdp 



E(Ri, . . . , R m ) e q / p (Ri, . . . ,R : 
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= f e:c (Ri,...,R m ) dx (12) 
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We can formulate a general criterion for belief accep- 
tance: when we have limited knowledge about the re- 
liability of our information sources, we can expand our 
belief set from {Ri, . . . , R n } to {Ri, . . . , R n +i} if and 
only if 



E(R 1 



, Rn+l) > E(R 1: . . . , R n ). 



(13) 



Bayesian Networks 

Bayesian Networks represent (conditional) indepen- 
dences between variables and when implemented on 
a computer they perform complex probabilistic calcu- 
lations at the touch of a keystroke. We are assuming 
here that the reader has some familiarity with Bayesian 
Networks (Cowell et. al. 1999, Jensen 1996, Neapoli- 
tan 1990, Pearl 1988). 

We construct a Bayesian Network that permits us 
to read off the reliability-relative coherence measure of 
an information set {Ri, . . . , R n } in Figure 1. First, we 
construct a Bayesian Network with nodes for the vari- 
ables Ri , . . . , R n which represents the marginal prob- 
ability distribution over these variables. Then we add 
nodes for the variables REPRi , . . . , REPR n and draw 
in an arrow from each node for the variable Ri to the 
node for the variable REPRi and specify the con- 
ditional probabilities in (|^) for each arrow. By the 
standard criterion of d-separation, we can now read 
off the conditional independences in (|l]) from the net- 
work. Subsequently, we construct a node for the vari- 
able & . . . &i?„: we draw in the arrows and specify 
conditional probabilities such that Ri & . . . &R n holds 
if and only if Ri , . . . , and R n hold. We can now read off 
P*(Ri, . . . , R n ): it is the probability of Ri& . . . &R n 
after instantiating REPRi, REPR n . To read 
off P max *(Ri, . . . , R n ), more construction is needed. 
Notice that P max {Ri) = P max {R 1 k . . . &R n ) for 
i = 1 , . . . , n in the counterfactual case of maximal co- 
herence, is equal to P(Ri& . . . &R n ) in the actual case 
where the information set may not be maximally co- 
herent. Hence P max (Ri, . . . , R n ) is the posterior joint 
probability of Ri , . . . , R n , had we been informed in the 
actual case by n less than fully reliable independent 
sources that Ri&...&R n . So we add nodes for the 
variables REPiSzR (whose positive values states that 




Figure 1: Bayesian Network for coherence measure 



P<R I &...&R„,IR 1 &...&R„, R, tl ) = 1 
P(R,&...&R„IR , R„) s 1 




Figure 2: Bayesian Network for belief expansion 



the z-th source informs us that Ri& . . . &R n ), draw in 
the proper arrows and specify the proper conditional 
probabilities. We can now read off p max * (R 1; . . . r R n ); 
it is the probability of Ri& . . . &R n after in- 
stantiating REPi&R, . . . , REP n &R. The measure 
c x (Ri& . . . &R n ) follows by (g). 

We construct a Bayesian Network in Figure 2 to 
determine whether belief expansion is warranted or 
not. The construction of the nodes for the variables 
Ri,..., R n+ i and REPRi , . . . , REPR n+1 should be 
clear from our construction of the Bayesian Network 
in Figure 1. This part of the Bayesian Network re- 
spects the conditional independences in (||). Now we 
add a node for the variable R±8z .. . SzR n and a node 
for the variable & . . . &i?„+i and specify the condi- 
tional probabilities so that Ri&...&R n holds if and 
only if Ri, . . . , and R n hold and Ri& . . . &R n +i holds 
if and only if Rj, . . . , R n and R n +i hold. We instan- 
tiate REPRi , . . . , REPR n and propagate the evidence 
throughout the network. We can now read off the ac- 
ceptance measure e x (Ri, . . . , R n ) which is the posterior 
probability of Ri & . . . &R n . To raise the question of 
belief expansion, this value should be greater than or 
equal to our threshold value for belief. Subsequently, 
we instantiate REPR n+ i and propagate the evidence 



throughout the network. We can now read off the ac- 
ceptance measure e^Ri, . . . , R n +i) which is the poste- 
rior probability of Ri & . . . &R n +i. Depending on our 
treshold value for belief, we can determine whether we 
are justified to expand our beliefs with the proposition 

Rn+l- 

It is easy to see how the idealizations can be re- 
laxed in the networks. We can stipulate alternative 
reliability parameters for the sources. We can let one 
source report on two items of information. We can 
add arrows between the REPRi variables or between 
some REPRi and Rj variables (for i ^ j) to model 
certain types of dependence between the sources. It 
suffices that -P(Ri, ■ ■ ■ , R n ) is equal to or greater than 
the threshold value for belief. Furthermore, even if 
P(Ri, . . . ,R n +i) is below the threshold value for be- 
lief, the model yields a marginal probability distribu- 
tion over Ri, . . . , R n . Hence, the general question of 
belief revision becomes a question of defining a func- 
tion which maps joint probability distributions over a 
set of propositional variables into sets of propositions 
that are values of a subset of these variables and that 
can reasonably be believed. Defining such a function 
is beyond the scope of this paper. 

Conclusion 

(i) We have designed a procedure to determine a par- 
tial coherence ordering over a set of information sets 
of size n. If one information set is more coherent than 
another on this ordering, then our degree of confidence 
in the content of the former set will be greater than in 
the content of the latter set, after having been informed 
by independent and less than fully reliable sources, ce- 
teris paribus, (ii) We have designed a probabilistic 
criterion for (non-prioritized) belief expansion, which 
determines whether it is rational to believe new infor- 
mation, considering how reliable the sources are and 
how well the new information coheres with the old in- 
formation, (iii) If either the sources are sufficiently re- 
liable or the information set sufficiently large, then the 
question of belief expansion is largely determined by 
whether the expanded information set is more coherent 
than the original information set (provided that there 
exists an ordering of this pair of information sets), and 
only marginally by the reliability of the sources, (iv) 
We have shown how a coherence ordering over infor- 
mation sets can be constructed by means of Bayesian 
Networks and how belief expansion can be modeled 
by means of Bayesian Networks in an empirically ade- 
quate manner. 
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